Fast messages: efficient, portable communication for workstation clusters and MPPs

Prognostications of the National Information Infrastructure’s future structure typically include high-performance servers of information, computation, and other specialized services embedded in a high-speed network fabric with hundreds of millions of other hosts. Two system architectures are likely candidates for these servers: massively parallel processors and networks of workstations. Both are attractive because of their ability to scale. However, both critically depend on internal communication performance to be effective servers, and any NII server depends on excellent external networking to support NII service. Interestingly, in recent years, MPP and NOW hardware have become increasingly similar, as both are driven by the significant cost advantages of high-volume products. Hence, many issues involved in delivering communication performance in such systems have converged as well. The Illinois Fast Messages project intends to exploit this convergence to develop communications technology that spans both MPPs and NOWs, supporting both intracluster communication and high-speed external networking. FM is a portable, low-level messaging interface that can deliver high network data rates, even for small messages. It delivers the lowlatency communication that is essential for efficient coordination and data movement on large-scale parallel systems. FM implementations not only deliver high performance but also provide the high-level guarantees that enable streamlined implementations of higher-level protocols atop FM. FM delivers performance to the higher-level layers, not just to applications written directly to the messaging layer. We’ve implemented FM 1.1 on the Cray T3D MPP and on Myrinetbased workstation clusters. Both implementations perform substantially better than the vendor-supplied messaging layers. We’ve also implemented two widely accepted standard interfaces—Unix sockets and the Message